Add datadog version tag configuration #16383

acrollet · 2024-04-17T17:47:08Z

Summary

This work is behind a feature toggle (flipper): NO

This PR adds configuration for Datadog version tagging. This will support various quality of life improvements in datadog, and functionality like faulty deployment detection.

Dependencies

This relies on https://github.com/department-of-veterans-affairs/vsp-infra-application-manifests/pull/2807, but can be committed before it is.

What areas of the site does it impact?

datadog service configuration

github-actions · 2024-04-17T17:47:48Z

	1 Error
🚫	This PR changes `1146` LoC (not counting whitespace/newlines). In order to ensure each PR receives the proper attention it deserves, those exceeding `500` will not be reviewed, nor will they be allowed to merge. Please break this PR up into smaller ones. If you have reason to believe that this PR should be granted an exception, please see the Submitting pull requests for approval - FAQ. File Summary Files .github/workflows/build.yml (+182/-5) .github/workflows/code_checks.yml (+0/-121) .github/workflows/deploy-template.yml (+98/-0) .gitignore (+1/-0) Dockerfile (+53/-96) Gemfile (+1/-1) Makefile (+1/-1) app/controllers/application_controller.rb (+0/-1) app/uploaders/uploader_virus_scan.rb (+4/-3) bin/deps (+1/-0) bin/fake_clamdscan (+0/-5) bin/test (+1/-0) bin/test-setup (+2/-0) clamav_tmp/1e7e39bfccbd672ca2fd00ead324b391 (+1/-0) clamav_tmp/a0960199b278fd7e95f4df40f56046cd (+1/-0) clamav_tmp/fe1ade8b718b283067640956690ffba6 (+1/-0) config/ca-trust/README.md (+5/-1) config/ca-trust/fwdproxy.crt (+24/-0) config/clamd.conf (+4/-2) config/freshclam.conf (+1/-1) config/initializers/clamav.rb (+18/-0) config/initializers/clamscan.rb (+0/-5) config/initializers/datadog.rb (+0/-4) config/settings.yml (+3/-0) docker-compose-clamav.yml (+10/-0) docker-compose-deps.yml (+8/-1) docker-compose.yml (+46/-34) docs/setup/hybrid.md (+40/-1) docs/setup/native.md (+41/-22) docs/setup/running_docker.md (+21/-0) docs/setup/running_natively.md (+19/-7) import-va-certs.sh (+5/-2) lib/clamav/commands/patch_scan_command.rb (+43/-0) lib/clamav/patch_client.rb (+76/-0) lib/common/file_helpers.rb (+17/-0) lib/common/models/redis_store.rb (+2/-2) lib/common/virus_scan.rb (+12/-4) lib/shrine/plugins/validate_virus_free.rb (+5/-3) spec/lib/shrine/plugins/validate_virus_free_spec.rb (+9/-23) spec/models/persistent_attachments/dependency_claim_spec.rb (+5/-2) spec/models/persistent_attachments/lgy_claim_spec.rb (+5/-2) spec/models/persistent_attachments/pension_burial_spec.rb (+5/-2) spec/requests/claim_documents_spec.rb (+2/-2) spec/simplecov_helper.rb (+0/-1) spec/spec_helper.rb (+0/-1) spec/support/uploader_helpers.rb (+1/-7) spec/uploaders/uploader_virus_scan_spec.rb (+3/-7) Note: We exclude files matching the following when considering PR size: `.csv, .json, .tsv, .txt, Gemfile.lock, app/swagger, modules/mobile/docs, spec/fixtures/, spec/support/vcr_cassettes/, modules/mobile/spec/support/vcr_cassettes/, db/seeds, modules/vaos/app/docs, modules/meb_api/app/docs, modules/appeals_api/app/swagger/, *.bru` Big PRs are difficult to review, often become stale, and cause delays.

Generated by 🚫 Danger

va-vsp-bot · 2024-04-17T17:47:59Z

Error: A file (or its parent directories) was deleted but its reference still exists in CODEOWNERS. Please update the .github/CODEOWNERS file and delete the entry for the Offending file:

va-vsp-bot · 2024-04-17T17:48:00Z

Error: A file (or its parent directories) does not have a CODEOWNERS entry. Please update the .github/CODEOWNERS file and add the entry for the Offending file:

va-vsp-bot · 2024-04-17T19:59:25Z

Error: A file (or its parent directories) does not have a CODEOWNERS entry. Please update the .github/CODEOWNERS file and add the entry for the Offending file:

va-vsp-bot · 2024-04-17T19:59:26Z

Error: A file (or its parent directories) was deleted but its reference still exists in CODEOWNERS. Please update the .github/CODEOWNERS file and delete the entry for the Offending file:

rmtolmach · 2024-04-18T20:17:53Z

.github/workflows/deploy-template.yml

@@ -81,6 +81,9 @@ jobs:
          for env in ${envs[*]};
          do
            yq e -i '(.vets-api.image.tag) |= "${{ github.sha }}"' $env/values.yaml
+            yq e -i '(."vets-api".web.envSecrets[] | select(.name=="DD_VERSION").value) |= "${{ github.sha }}"' $env/values.yaml
+            yq e -i '(."vets-api".web.dbMigrate.envSecrets[] | select(.name=="DD_VERSION").value) |= "${{ github.sha }}"' $env/values.yaml
+            yq e -i '(."vets-api".worker.envSecrets[] | select(.name=="DD_VERSION").value) |= "${{ github.sha }}"' $env/values.yaml


What's the reason for having "vets-api" in quotes on these three lines?

It was necessary because of the hyphen when I was testing locally, it would probably be good to add it to the existing line as well. Maybe it's due to a yq version difference?

possibly. I bet both versions work.

The jq manual says that keys with special chars should be quoted - I went ahead and added it to the first line for consistency.

ok sounds good!

rmtolmach · 2024-04-18T20:19:35Z

config/initializers/datadog.rb

@@ -9,6 +9,7 @@
    # Namespace our app
    c.service = 'vets-api'
    c.env = Settings.vsp_environment unless ENV['DD_ENV']
+    c.version = AppInfo::GIT_REVISION unless ENV['DD_VERSION']


I'm guessing we don't also need this on the master branch of this file as well?

We will definitely need this in master as well.

va-vsp-bot · 2024-04-18T21:19:53Z

Error: A file (or its parent directories) was deleted but its reference still exists in CODEOWNERS. Please update the .github/CODEOWNERS file and delete the entry for the Offending file:

va-vsp-bot · 2024-04-18T21:19:55Z

Error: A file (or its parent directories) does not have a CODEOWNERS entry. Please update the .github/CODEOWNERS file and add the entry for the Offending file:

va-vsp-bot · 2024-04-18T21:29:20Z

Error: A file (or its parent directories) does not have a CODEOWNERS entry. Please update the .github/CODEOWNERS file and add the entry for the Offending file:

va-vsp-bot · 2024-04-18T21:29:23Z

Error: A file (or its parent directories) was deleted but its reference still exists in CODEOWNERS. Please update the .github/CODEOWNERS file and delete the entry for the Offending file:

va-vsp-bot · 2024-04-18T21:31:06Z

Error: A file (or its parent directories) was deleted but its reference still exists in CODEOWNERS. Please update the .github/CODEOWNERS file and delete the entry for the Offending file:

va-vsp-bot · 2024-04-18T21:31:13Z

Error: A file (or its parent directories) does not have a CODEOWNERS entry. Please update the .github/CODEOWNERS file and add the entry for the Offending file:

rmtolmach

Tests passing. The ❌ failures are expected and can be ignored.

acrollet added 2 commits April 17, 2024 17:32

Add datadog version config

73ae737

Update datadog version tag in manifest on deploy

3ded666

acrollet requested review from a team as code owners April 17, 2024 17:47

va-vfs-bot had a problem deploying to dd-version-vets-api/main/main April 17, 2024 17:59 Error

Merge branch 'k8s' into dd-version-vets-api

30dace8

va-vfs-bot had a problem deploying to dd-version-vets-api/main/main April 17, 2024 20:30 Error

rmtolmach reviewed Apr 18, 2024

View reviewed changes

Remove datadog initializer change from k8s branch

5069b84

Update values editing for consistency

b3b887a

Merge branch 'k8s' into dd-version-vets-api

b4ba315

rmtolmach approved these changes Apr 18, 2024

View reviewed changes

va-vfs-bot had a problem deploying to dd-version-vets-api/main/main April 19, 2024 11:44 Error

rmtolmach merged commit 4c70618 into k8s Apr 19, 2024
15 of 18 checks passed

rmtolmach deleted the dd-version-vets-api branch April 19, 2024 19:37

rmtolmach mentioned this pull request Apr 19, 2024

Implement Semantic-Version Tool for vets-api department-of-veterans-affairs/va.gov-team#77542

Open

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add datadog version tag configuration #16383

Add datadog version tag configuration #16383

acrollet commented Apr 17, 2024 •

edited

Loading

github-actions bot commented Apr 17, 2024 •

edited

Loading

Files

va-vsp-bot commented Apr 17, 2024

va-vsp-bot commented Apr 17, 2024

va-vsp-bot commented Apr 17, 2024

va-vsp-bot commented Apr 17, 2024

rmtolmach Apr 18, 2024

acrollet Apr 18, 2024

rmtolmach Apr 18, 2024

acrollet Apr 18, 2024

rmtolmach Apr 18, 2024

rmtolmach Apr 18, 2024

acrollet Apr 18, 2024

va-vsp-bot commented Apr 18, 2024

va-vsp-bot commented Apr 18, 2024

va-vsp-bot commented Apr 18, 2024

va-vsp-bot commented Apr 18, 2024

va-vsp-bot commented Apr 18, 2024

va-vsp-bot commented Apr 18, 2024

rmtolmach left a comment

Add datadog version tag configuration #16383

Add datadog version tag configuration #16383

Conversation

acrollet commented Apr 17, 2024 • edited Loading

Summary

Dependencies

What areas of the site does it impact?

github-actions bot commented Apr 17, 2024 • edited Loading

Files

va-vsp-bot commented Apr 17, 2024

va-vsp-bot commented Apr 17, 2024

va-vsp-bot commented Apr 17, 2024

va-vsp-bot commented Apr 17, 2024

rmtolmach Apr 18, 2024

Choose a reason for hiding this comment

acrollet Apr 18, 2024

Choose a reason for hiding this comment

rmtolmach Apr 18, 2024

Choose a reason for hiding this comment

acrollet Apr 18, 2024

Choose a reason for hiding this comment

rmtolmach Apr 18, 2024

Choose a reason for hiding this comment

rmtolmach Apr 18, 2024

Choose a reason for hiding this comment

acrollet Apr 18, 2024

Choose a reason for hiding this comment

va-vsp-bot commented Apr 18, 2024

va-vsp-bot commented Apr 18, 2024

va-vsp-bot commented Apr 18, 2024

va-vsp-bot commented Apr 18, 2024

va-vsp-bot commented Apr 18, 2024

va-vsp-bot commented Apr 18, 2024

rmtolmach left a comment

Choose a reason for hiding this comment

acrollet commented Apr 17, 2024 •

edited

Loading

github-actions bot commented Apr 17, 2024 •

edited

Loading